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I. Introduction 

This paper describes the validation of one aspect of the Value Creation Model (VCM) that Lucent's Ad- 
vanced Technology (AT) group has developed and applied to research driven innovation projects. This aspect is the 
Rate/Productivity Metric (RPM) that is used to quantify and understand the expected present value of financial re- 
turn of each project. The RPM includes estimates of the future dollar returns along with the risk associated with 
these returns thereby enabling the calculation of an expected value of returns. Uniform use of this metric allows 
management to analyze the collection of projects uniformly as a portfolio system which aids in the decision making 
purposes. 

RPM is a metric and in order for any metric system to assist a decision maker, the metric must be able to 
demonstrate its validity to management. For a risk metric, one necessary validity test is the correlation between risk 
and corresponding reward. This is especially true of a metric design to delineate the value provided to the corpora- 
tion by a portfolio of research driven innovation projects. 

Risk in R&D can come from multiple sources such as technology, staff capability, marketing, and resource 
availability/commitments. The RPM, in its present implementation, measures risk as a summary estimate of all 
these sources combined. Given the summary form, validity of the RPM needs to be established to assure the devel- 
opers and management that the metric has been properly developed. 

Any investment risk model is expected to show a positive relationship between the magnitude of expected 
payoff and the degree of risk. Unless such a correlation can be demonstrated, the metric is not likely to be useful as 
input to the management process. 



II, Background 

The development of an appropriate produc- 
tivity measure for research driven innovation projects 
and the ability of managers to apply that measure 
meaningfully to a portfolio of projects so as to accu- 
rately portrays the R&D risk reward frontier has 
proved elusive. Many firms revert to relying on short 
term financial measures that evaluate the perform- 
ance of their organization in terms of net present 
value. For long lead time R&D projects, the discount 
factors on returns expected in five to 15 years from 
now weaken the use of such conventional tools. 
Other firms simply forgo the effort to develop 
meaningful measures to support their R&D proc- 
esses. Still others have reacted by radically changing 
the charter of their R&D organization or by elimi- 
nating their internal R&D function altogether. 



These extremes are fueled on one side by 
those who suggest that no short term financial metric 
is suitable and therefore cannot aid in managing any 
portion of the R&D process. Simple stated "ROI is 
not enough" (Mechlin and Berg 1980). These practi- 
tioners and academics argue that the R&D process is 
too eclectic, very difficult to meaningfully measure 
on any basis and difficult if not impossible to justify 
by financial analysis alone. These professionals per- 
ceive that calculating discounted cash flow or ROI in 
order to manage the R&D process more effectively is 
simply adding guess work on top of uncertainty. 
Given these assumptions, it is not unusual for R&D 
organizations to operate as cost centers and are 
treated as such in management decision making. 

Others counter with the quality dictum "If 
you can't measure it, you can't manage it If data 
does not exist to support a measure, the management 
process for a key strategic objective is likely to be 
inadequate or non-existent" (Endres 1997). Further, 
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they argue that the language of management is 
money and that sooner or later R&D must measure 
up to that same standard. This too has a ring of truth 
but the operative questions that guide the manage- 
ment of the research driven innovation process must 
become: What should we be measuring? How should 
these factors be measured? Can these measures use- 
fully be applied across a portfolio of R&D projects? 
And: "How can financial risk and reward be meas- 
ured within our management model?" 

Lucent Technologies has faced this problem 
squarely and dedicated a team of Bell Laboratories 
scientists to the task of creating a decision support 
system that allows management to measure and make 
informed decisions about its Advanced Technologies 
(AT) Group R&D operations. The overall model is 
called the Value Creation Model. One component of 
this model is the Risk/Productivity Metric (RPM). 
This paper describes the nature of the RPM and how 
it was validated. 

This paper displays our model and our met- 
ric. However, it focuses on the developed and tested 
RPM metric portion in order to discuss the relation- 
ship between risk and reward in the decision support 
system utilized to assist in the management of R&D 
at Lucent's Bell Labs AT group. Our model and 
metric has been applied to a portfolio of 522 projects 
that provide over two years of data. We first provide 
a visual display of our interactive model We then 
discuss the RPM metric and our summary measure of 
risk and finally we test for the relationship between 
risk and reward in our portfolio of research driven 
innovation projects at Lucent's AT division. 

Briefly the measurement of risk in this deci- 
sion support system is made by computing the vari- 
ance of future expected returns. Our risk measure 
utilizes a standard deviation measure of risk but has 
interactive real time visual links allowing the deci- 
sion maker to investigate various parameter interac- 
tions in order to understand the portfolio's adherence 
to an efficient risk reward frontier. This can be done 
for either the portfolio of projects, segment thereof, 
or any particular project in that portfolio. In our 
model risk is expressed numerically but modified by 
the visual interactions that allow the manager to alter 
the focus and type of projects in the portfolio. This 
enables subset portfolio risk reward paradigm analy- 
sis. 

HI. The VCM Model 

The Lucent Value Creation Model (VCM) 
for AT projects possesses five characteristics and 
four primary attributes which embody many of the 



key findings that R&D researchers have identified as 
important. The essence of the model is a decision 
support tool that provides visual displays of the key 
factors of the research development process. The 
VCM provides no "right answers" but rather allows 
decision makers to visualize the interactive relation- 
ships among projects and their attributes through a 
interactive visual display of many complex and com- 
peting management imperatives. 

The objectives guiding the creation of the 
VCM model are: 

" The model must have a forward looking focus, 
not past; 

» The process is to be used to baseline and set ob- 
jectives; 

■ The model must be easy to understand to all 
members of the organization; 

■ The process must link to corporate strategies; 

■ The model must include important factors asso- 
ciated with all the objectives of R&D such as 
developing leading edge technologies or com- 
petence development. 

These goals led to the definition of four key 
factors that need to be explicit in the RPM metric: 
cost revenue, time and risk. In addition a set of five 
categories of attributes and other factors that opera- 
tionalize the organization's goals were defmed. 
These attribute categories and attributes are; 

■ Business units - The Lucent Business Unit(s) 
involved with each project are identified. 

■ Intellectual property - Two aspects of the intel- 
lectual property characteristics of each projects 
are measured, 

■ Techno logies - Two measures of the character of 
the technologies applied are measured. 

■ Markets - The market addressed by the project 
as well as the life cycle of the product are cap- 
tured. 

« Strategic initiatives - Each project is related to 
Lucent's strategies. 

The selection of these characteristics, factors 
and attributes provides a path for measuring R&D 
quality. As noted by Enders (1997) " . . measure- 
ment pyramids for R&D organization include: port- 
folio evaluation, numbers/ quality/ functionality of 
resources." These goals link R&D directly to the 
overall corporate strategy providing a: "Balanced 
scorecard perspective: financial, customer, internal 
business processes, learning and growth." (Kaplan 
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1996) Further, the implementation of this model has 
helped InitidLtG a cultural change at Lucent AT. Ron 
Jetko a CFO at Lucent AT states "...R&D labs' sci- 
entists are now required to think in terms of research 
projects' relevance to corporate strategies and their 
projected contributions to the bottom line" 

The VCM system allows management to 
easily identify these projects and ask important R&D 
planning questions such as: Should project funding 
be shifted toward the high-value low-risk quadrant? 
Are too many projects in the low risk, low value 
quadrant? Admittedly, there are many considerations 
involved in answering these questions. But, the 
VCM allows Lucent management to visualize the 
relationships among the nine attributes described 
above as part of the planning process. 

Figure 1 shows one view of the VCM visual 
outputs that displays output relationships of ten proj- 
ects across eight attributes. Here each column identi- 
fies all the projects. The linkages as shown by the 
yellow lines and nodes are the specific subset of 



Figure 1 VCM Visual Query 

projects we are inquiring after. Each column is an 
attribute and each line is a project. The area of the 
circles describes characteristics of the attribute. For 
example, under product life cycle, the relative sizes 
of the circles represent the distribution of the projects 
in the portfolio by product life cycle. 

This model does not make a decision. 
However, the VCM promotes inquiry and stimulates 
discussion that focuses management attention on 
multiple attributes rather than myopicaily using ROI 
as the sole decision criteria. Nonetheless, the RPM 
can be accessed separately to provide a comparison 
of expected financial return among projects. 



IV. The Risk/Productivity Metric 

The RPM portfolio metric is a simple bene- 
fit-cost ratio. It is the ratio of expected future incre- 
mental cash flows resulting from the project attrib- 
uted to the incremental R&D expense. The cash in 
flows and expense out flows are computed at the cor- 
porate level. The metric is named the "Risk Portfolio 
Metric" or RPM. 

(1) RPM ^NPV (Net corporate incremental 
cash flow ) /NPV(R&D 's incremental invest- 
ment) 

A. Risk Calculation 

We measure the risk in our portfolio and 
quantify the riskiness of the individual projects in the 
portfolio through a series of project generated cash 
flow estimates. Project managers prepare a pessi- 
mistic, realistic and optimistic estimate of cash flows 
from each project. The three estimates are used to 
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establish the parameters of a triangle distribution that 
is then used as the measure of risk. For purposes of 
comparing across projects, individual project risk is 
computed as the normalized standard deviation of the 
triangle distribution (Winston 1996). The character 
of risk in Lucent AT's risk can is visually displayed 
by the histogram projected by the VCM shown in 
Figure 2. Here again we can compare our selected 
ten projects in yellow with respect to the portfolio as 
a whole. The x axis is the normalized standard de- 
viation and the y axis is number of projects. 

B. Testing the Model: The Risk Reward Relation- 
ship 

Using our definition of project risk and the 
project RPM as a proxy for expected project return or 
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corporate value we can construct a basic RPM-risk 
scatter plot. We check to determine if our project 

Figure 2: Histogram of Risk Normalized Standard 
Deviation vs Number of Projects 



portfolio supports the fundamental theory of financial 
portfolio management which roughly states that ex- 
pected return is directly proportional to the risk of the 
return (Brealyl980). We accomplish this by re- 
gressing the project RPM values against the project 
risk. A positive regression coefficient with a strong 
t-statistic is an indication that in general the riskier 
projects in our portfolio are expected to have the 
higher returns. 

The regression had a t-statistic value of 4.43 
and an F-statistic of 19.60. The t-statistic demon- 
strates that the regression coefficient is positive at the 
99% level while the F-statistic demonstrates that the 
coefficient is non-zero at the 99% level and thus has 
significant explanatory power. The R-squared value 
of .04 is somewhat low due to the wide spread of 
RPMs among the low risk projects and the different 
methods utilized to compute cash flows. Such a dis- 
tribution among low risk projects is not inconsistent 
with the AT Group's operations. Many of AT's 
projects 



are customer driven development projects where 
customers commit to the payment in advance so that 
the expected returns are known in advance and the 
variances among the pessimistic, realistic and opti- 
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mistic estimates are quite small. Nonetheless, we are 
investigating these projects more carefully to im- 
prove our understanding of how they can be properly 
represented in an otherwise risky portfolio. 

Our results are depicted in Figure 3, which 
presents a scatter plot of the portfolio projects in 
RPM-risk space. It also shows the results of re- 
gressing the project RPM value by project risk. 

V. Conclusions 

The current implementation of the risk met- 
ric appears to be appropriate for management use in 
that it is valid and easily understood. As such, it 
proves to be a useful tool for assessing the viability 
of existing projects and evaluation and choice among 
new projects. It provides a mechanism to measure 
and managed AT's R&D processes. 
Figure 3: Scatter Diagram and Regression Line of 
RPM vs Risk 
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Figure 3: Scatter Diagram and Regres- 
sion Line of RPM Vs. Risk* 

However, given the extensive literature on 
alternative types of risk, the risk metric appears to be 
conceptually simplistic. Incorporating more sophisti- 
cated measures of risk may reduce the currently high 
degree of unexplained error variance (low r 2 ). We 
are exploring several additional risk evaluation tech- 
niques. See figure Three. 
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